The invention relates to a method for correcting the gray levels or pixel intensities of images-produced by a digital infrared camera having a two-dimensional infrared detector array, wherein the gray level value for each image pixel is corrected by a stored correction coefficient.
The German company AEG Infrarot Module GmbH (AIM) has been producing high grade infrared detectors and associated components such as coolers and output electronics since the 1970s. Such detectors are typically used in military, research, medical, and industrial applications. In older one-dimensional or line detectors, a one-dimensional line of photosensitive detector elements cooperated with a mechanical scanner so as to successively scan a scene line-by-line so as to generate a complete image therefrom. In the newest generation of detectors, the individual photosensitive elements are arranged in a two-dimensional array over a surface plane of the detector. Since the array of elements directly receives a complete image over the two-dimensional array plane, such two-dimensional detectors no longer need a mechanical scanner for generating a complete two-dimensional image. In other words, using a two-dimensional detector, a scene can be directly imaged onto a photosensitive array chip using a suitable optics system. The respective output signals of the individual photosensitive elements are serially read-out and then reconstructed into a two-dimensional data structure representing the corresponding image, by means of a suitable electronic image processing circuit and procedure.
Due to technical reasons and limitations, any two-dimensional infrared detector comprising a two-dimensional array of individual photosensitive elements necessarily exhibits inhomogeneities or non-uniformities among the several individual elements. These non-uniformities can include constant or invariable non-uniformities as well as time-varying non-uniformities. In any event, such non-uniformities cause imaging errors or aberrations in the electronic capture and reproduction of the true or actual image. This arises because a difference in the output signal of a given detector element relative to the outputs of the other elements would initially be regarded as arising from a feature of the real scene being viewed, even though such a difference might actually arise do to a non-uniformity of the respective detector element.
The problem of distinguishing between scene-based features and detector-based non-uniformities is most readily apparent when all of the detector elements are detecting a neutral uniform scene without any characteristic features. In such a case, it can be assumed that any difference in output signals of the several detector elements arises from detector-based non-uniformities. However, the imaging errors resulting from non-uniformities become especially problematic and difficult to recognize and correct when the detector is xe2x80x9cseeingxe2x80x9d or imaging a scene that includes features having different image pixel intensities, because then the detector elements will produce respective differing output signal intensities resulting from both the true differences in the scene as well as the non-uniformities of the individual detector elements. In this regard, features in the scene that move relative to the detector can be recognized as such real features, because the variation in pixel intensity associated with such a moving feature will move across several detector elements. On the other hand, stationary scene-based features will constantly affect the output intensity of a particular one or more detector elements of a stationary detector, and the output signals are therefore prone to be incorrectly interpreted as resulting from a detector-based non-uniformity in the respective affected element or elements.
European Patent Application 0,600,742, published on Jun. 8, 1994, and corresponding U.S. Pat. No. 5,323,334 (Meyers et al.), issued on Jun. 21, 1994 discuss the above mentioned problem relating to the recognition and correction of imaging errors resulting from non-uniformities among the individual photosensitive elements of a detector or sensor array. The entire disclosure of U.S. Pat. No. 5,323,334 is incorporated into the present application by reference, excepting any subject matter that may be further incorporated by reference into U.S. Pat. No. 5,323,334.
EP 0,600,742 and U.S. Pat. No. 5,323,334 also both disclose a system for carrying out such correction or suppression of imaging errors resulting from sensor element non-uniformities. In the known system, the sensor or detector array is mounted on a sensor positioner, which physically moves the sensor relative to the incident image of the external scene falling on the sensor. The sensor is physically moved in order to distinguish between detector-based non-uniformities on the one hand and features in the real scene being viewed by the sensor on the other hand. As the sensor is moved by the sensor positioner, a sensor-based non-uniformity will move with the sensor, i.e. will always remain associated with a particular element of the sensor, while a real feature in the scene will move relative to the sensor, i.e. will successively fall on different elements of the sensor (unless the feature happens to follow exactly the same motion as the sensor, which can be avoided by proper choice of the motion pattern). A non-uniformity compensator is the used to recognize and correct or compensate the detector-based non-uniformities by processing the output signal of the sensor. Thereafter, a position corrector circuit electronically corrects the output signals by the inverse or opposite of the physical motion of the sensor, so that the image information is electronically shifted back to its true position, i.e. to counteract the physical motion of the sensor.
While the known system of the above discussed references uses a valid concept or theory for recognizing and correcting sensor-based non-uniformities, problems arise in connection with the physical movement of the sensor itself. The mechanical actuators, gimbals and the like used for mounting and moving the sensor are subject to wear as well as mechanical tolerances and inaccuracies. Also, especially for larger sized sensor arrays, the size and mass of the sensor array makes it difficult to accurately and rapidly physically move the entire array.
German Patent 197 15 983, published on Sep. 24, 1998 and corresponding U.S. patent application Ser. No. 09/061,147, which is cross-referenced and incorporated by reference herein, are not prior art relative to the present application. Instead, the present application represents a further development that is preferably used in combination with the method and apparatus for correcting the gray levels of images produced by a digital infrared camera by means of electronic signal processing as disclosed in German Patent 197 15 983 and corresponding U.S. Pat. application Ser. No. 09/061,147.
In the system according to German Patent 197 15 983 and U.S. patent application Ser. No. 09/061,147, a signal processing arrangement includes a memory in which respective correction coefficients K for each detector element j of the detector are stored, and the associated signal processing method comprises the following steps. First, the respective gray levels or pixel intensities Uj of a scene are acquired as image data by the detector. Next, the gray levels Uj of the image are corrected through use of the correction coefficients K in the image processing system, and the resulting corrected gray levels Uke are intermediately stored. Then the corrected gray levels UkG are filtered through a locally effective adaptive filter M, to produce filtered corrected gray levels F(Ukj). The remaining image error or aberration ej is then determined by using the gray levels Ukj of the unfiltered corrected image and the gray levels F(Ukj) of the filtered corrected image. Then the correction coefficients K are progressively updated or improved in connection with this determined remaining image error ej. Finally, the improved or updated correction coefficients K are stored in the corresponding memory of the image processing system.
The system according to German Patent 197 15 983 and U.S. patent application Ser. No. 09/061,147 is generally effective for correcting images from a digital camera in order to compensate for the above described non-uniformities. However, that system by itself, does not overcome the problem that a stationary object or feature in the real external scene having a high spatial frequency or sharp image variation characteristic, such as a distinct edge for example, is continuously recognized and resolved by the correction method and thus has an influence on the updating of the correction coefficients. As a result, the updated correction coefficients have a tendency to compensate for such stationary objects or features in the scene, which has two consequences. First, such stationary features are progressively masked or compensated out of the resulting image. Secondly, when the detector thereafter views a different scene, e.g. when the detector moves or when the previously stationary features move, an inverse or ghost image of the stationary features will be produced in the resulting image due to the compensating effect of the updated correction coefficients.
In view of the above, it is an object of the invention to provide a method and-an apparatus for correcting the gray levels or pixel intensities of-images produced by a digital infrared camera, which are improved so that stationary features in a scene do not have an influence on the updating of the correction coefficients. Particularly such a method and apparatus should avoid the progressive masking of stationary features of a scene and the formation of ghost-images associated with such stationary features. It is a further aim of the invention to achieve, in combination, all of the advantages of the above discussed methods and apparatuses, while avoiding or overcoming the disadvantages of the prior art.
The above objects have been achieved according to the invention in a method for correcting the gray levels or pixel intensities of images produced by a digital infrared camera having a two-dimensional detector. According to the inventive method, correction coefficients Kj for each respective detector element j are stored in a memory of an image processing system, and these correction coefficients Kj are continuously updated or improved by means of a dynamic correction process. The progressively improved correction coefficients are used to progressively remove the influence of non-uniformities in the image data. Further according to the method, the image of the scene incident on the detector and/or the detector are moved relative to each other. Particularly according to the invention, the incident image is moved relative to the detector, while the detector itself is not physically moved. This is preferably achieved by providing a microscanner in the incident optical path of the infrared detector so that the microscanner selectively and controllably directs the image onto the detector and moves the image relative to the detector.
Due to the motion of the image on the detector, the location of image features falling on the detector will move among the several detector elements, so that the image processing can distinguish a stationary feature of the scene from a detector-based non-uniformity. Thus, the above described correction coefficients can be progressively updated or improved without being affected by the stationary features in a scene, and especially such stationary features having a high image variation frequency like a distinct edge, for example. Also, the use of a microscanner allows the detector to be stationary fixed in the camera, which simplifies the structure and use of the apparatus and method.
It is further provided that the incident image and the detector are preferably moved cyclically relative to each other, i.e. the image is moved cyclically relative to the detector, so that this relative motion can be easily achieved by any periodic motion arrangements that are typically known in scanners. Also in this case, the evaluation and image processing can be carried out in a correspondingly simple manner.
According to a further advantageous embodiment of the inventive method, the relative movement of the image on the detector has a magnitude or distance corresponding to a fractional portion of the spacing between the centers of neighboring ones of the individual elements of the detector as measured in the direction of the relative motion between the image and the detector. It is especially advantageous if the magnitude or distance of the relative motion corresponds to one half of the spacing between the centers of the detector elements in the relative motion direction. In this manner it is possible to carry out a suitable processing of the image data to achieve an effective doubling of the spatial resolution of the image in comparison to the actual physical spatial resolution of the detector.
A further embodiment feature of the invention provides that the cyclical motion of the image relative to the detector is carried out in a sequence of positions so that positions between respective closest neighboring detector elements are taken up first and then positions between more-distant detector elements are taken up in sequence. Such a particular path or pattern of the motion of the image on the detector serves to optimize, i.e. minimize, the time between respective successive partial images or fields that will be combined or interlaced to provide a complete image or frame of higher resolution.
It is especially preferred that four respective partial images or fields corresponding to four respective positions of the image incident on the detector are combined, e.g. interlaced, to form a single complete image or frame having a doubled spatial resolution. In this context, the correction coefficients are applied to each partial image or field to carry out the above mentioned correction before the fields are combined to form the complete image or frame.
The above objects have further been achieved according to the invention in a digital infrared camera including a two-dimensional infrared sensitive detector, an image processing system, and a microscanner arranged in the incident, optical path of the infrared camera. The image processing system includes a memory in which respective correction coefficients Kj for each detector element j of the detector are stored, and a processing circuit in which a dynamic correction process is carried out to continuously update or improve the stored values of the correction coefficients Kj. The microscanner is adapted and arranged to controlledly move the image incident on the detector relative to the detector by suitably deflecting the incident beam of infrared radiation.